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Abstract: This paper is a study of the polyhedral geometry of Gelfand-Tsetlin 
patterns arising in the representation theory gl„C and algebraic combinatorics. 
We present a combinatorial characterization of the vertices and a method to cal- 
culate the dimension of the lowest-dimensional face containing a given Gelfand- 
Tsetlin pattern. 

As an application, we disprove a conjecture of Berenstein and Kirillov 
about the integrality of all vertices of the Gelfand-Tsetlin polytopes. We can 
construct for each n > 5 a counterexample, with arbitrarily increasing denom- 
inators as n grows, of a non-integral vertex. This is the first infinite family of 
non-integral polyhedra for which the Ehrhart counting function is still a polyno- 
mial. We also derive a bound on the denominators for the non-integral vertices 
when n is fixed. 



1 Introduction 

Many authors have recently observed that polyhedral geometry plays a special 
role in combinatorial representation theory (see for example [21E1E1E1E]> an d 
the references within). In this note we study the polyhedral geometry of the 
so-called Gelfand-Tsetlin patterns, which arise in the representation theory of 
g[„C and the study of Kostka numbers. 

For each n £ N, let X n be the set of all triangular arrays (xij)i<i<j< n 
with Xij S R. Then X n inherits a vector space structure under the obvious 
isomorphism X n = R ( 2 + ' . 

Definition 1.1. A Gelfand-Tsetlin pattern or GT-pattern is a triangular array 
(xij)i<i<j<n £ X n satisfying the inequalities 
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• Xij > 0, for 1 < i < j < n; and 

• > Xy > ajj-i-ij+i, for l<i<j<n — 1. 

We always depict a GT-pattern (aJy)i<t<j'<n by arranging the entries as 
follows: 

^ln * * X nn 

Xl3 X 2 3 X 33 

X\2 X 2 2 
Xll 

In this arrangement, the inequalities in Definition 11.11 state that each entry is 
non-negative, and each entry not in the top row is weakly less than its upper- 
left neighbor and weakly greater than its upper-right neighbor. We refer to the 
elements x±j, . . . , Xjj as the jth row, i.e., the jth row counted from the bottom. 

The solutions of these inequalities define a polyhedral cone in R 2 . See the 
top of Figure for an example of a GT-pattern. 

Definition 1.2. Given X, p, € Z n , the Gelfand-Tsetlin polytope GT(A,/i) C X n 
is the convex polytope of GT-patterns (xij)i<i<j< n satisfying the equalities 

• Xi n = Xi, for 1 < i < n; 

• xn = Mi; and J2l=i x ij - Si=i x i,j-i = Mi) for 2 < j < n. 

In other words, GT(A,/i) is the set of all GT-patterns in X n in which the top 
row is A and the sum of the entries in the jth row is Yli=i Mj f° r 1 3 ^ n - 
Note that when we speak of a GT-polytope GT(A,/z), we assume that A and \i 
are integral. 

The importance of GT-polytopes stems from a classic result of I. M. Gelfand 
and M. L. Tsetlin in which states that the number of integral lattice points 
in the Gelfand-Tsetlin polytope GT(A, /z) equals the dimension of the weight 
\i subspace of the irreducible representation of Ql n C with highest weight A. 
These subspaces are indexed by the set SSYT(X, fi) of semi-standard Young- 
tableaux with shape A and content pi |18| . It is well-known that the elements 
of SSYT(\, /i) are in one-to-one correspondence with the integral GT-patterns 
in GT(\,p) under the bijection exemplified in Figure^ Given an integral GT- 
pattern in X n , let A^' be the jth row (so that A*-"-* = A). For 1 < j < n, place 
j's in each of the boxes in the skew shape X^'/X in the Young diagram 
of shape A. (Here we put A^ ' = to deal with the j = 1 case.) See QH] 
for details and EJ for more interesting uses of Gelfand-Tsetlin polytopes. 
Now we introduce the main combinatorial tool for the study of vertices of the 
Gelfand-Tsetlin polytopes: 
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Figure 1: A bijcction mapping GT(A,/i) n z("(™ +1 )/ 2 ) -» SSY~T(A,^) 



Definition 1.3. Given a GT-pattern x £ X n , the tiling 2? of x is the partition 
of the set 

{{%,]) £l?: l<i<j<n} 

into subsets, called tiles, that results from grouping together those entries in x 
that are equal and adjacent. More precisely, is that partition of {(i,j) £ 
Z 2 : 1 < i < j < n} such that two pairs (i, j), are in the same tile iff there 
are sequences 



ti, «2, ...,i r 



3 = 3i, 32, 



, 3r = 3 



such that %i k+1 j k+1 — %i k j k and 



{h+i,3k+i) G {(*fc + 1,3k + 1), («fc, ife + 1), {h - 1,3k - 1)j (ifei ife - 1)}, 
for each fc e {1, . . . , r — 1}. 



Given a GT-pattern x with tiling we associate to ,9 s (or, equivalcntly, 
to x) a matrix A&> as follows. Define the free tiles Pi, P2, . . . , P s of to be 
those tiles in & that do not intersect the bottom or top row of x, i.e., those 
tiles that do not contain (1,1) and do not contain (i,n) for 1 < i < n. The 
order in which the free tiles are indexed will not matter for our purposes, but, 
for concreteness, we adopt the convention of indexing the free tiles in the order 
that they are initially encountered as the entries of x are read from left to right 
and bottom to top. Define the tiling matrix A^> = (djk)2<j<n-i, i<k<s by 

ajk = #{«: e P k }. 
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Figure 2: Tilings of GT-patterns 



(Note that the index j begins at 2.) That is, a,jk counts the number of entries 
in the jth row of x that are contained in P^. 

Example 1.4. Two GT-patterns and their tilings are given in Figure [21 The 
unshaded tiles are the free tiles. The associated tiling matrices are respectively 

"1 1 0" 
1110 
10 1 

and 

1 

1 1 

2 2 0' 
111 

The motivation for introducing tilings, and the main result of this paper, is 
the following. 

Theorem 1.5. Suppose that is the tiling of a GT-pattern x. Then the dimen- 
sion of the kernel of Age is equal to the dimension of the minimal (dimensional) 
face of the GT-polytope containing x. 

As a corollary to this result, we get an easy-to-check criterion for a GT- 
pattern being a vertex of the GT-polytope containing it. 

Corollary 1.6. 7/x 6 GT(\,/i) has tiling containing s free tiles, then the 
following conditions are equivalent: 
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• x is a vertex of GT(A, /i); and 

• Agt has trivial kernel; i.e, for some s x s submatrix A of A&>, det A ^ 0. 

As an application of Theorem ll.5l we present a solution to a conjecture by 
Berenstein and Kirillov (p. 101, Conjecture 2.1 in £Q): all vertices of a Gelfand- 
Tsetlin polytope have integer coordinates, i.e. GT(A, \x) is a convex integral 
polytope. This conjecture seems to have been motivated by the fact that, for 
an integer parameter I, the Kostka number Kix,i^ is a polynomial in I. This 
was proved by Kirillov and Reshetikin using some fermionic formulas in [?]. 
More recently, Billey et al. [3j presented a more direct proof. For complete- 
ness, we give another proof at the end of Section [21 Derksen and Weyman 0] 
have now extended this polynomiality to other Ehrhart functions, the so-called 
Littlewood- Richardson coefficients. 

We must comment that it is quite natural to conjecture integrality of the 
vertices of GT-polytopes, if one knows of the theory of Ehrhart functions that 
count the number of lattice points inside convex polytopes and their dilations 
(see Chapter 4 |17|). The Ehrhart counting functions are known to be polyno- 
mials when the vertices are integral. As a consequence, in the following theorem 
we are in fact presenting the first infinite family of non-integral polyhedra whose 
Ehrhart counting functions are still polynomials. Other low-dimensional fami- 
lies have been found recently Finally, we must remark that R.P. Stanley 
communicated to us that his student Peter Clifford noticed non-integrality for 
GT-polytopes earlier (unwritten) and that King et al. had independently no- 
ticed non-integrality for hive polytopes (which generalize the GT-polytopes) in 
the case n = 5 (see |S]). They also proved integrality of vertices for n < 4, did a 
nice study of "stretched" Kostka coefficients, and presented several conjectures 
concerning again polynomiality of Ehrhart counting functions. 

Theorem 1.7. The Berenstein-Kirillov conjecture is true for n < 4, but coun- 
terexamples to this conjecture exist for all values of n > 5. More strongly, by 
choosing n sufficiently large, we can find GT-polytopes in which the denom- 
inators of the vertices are arbitrarily large: For positive integer k, let A = 
(fc fe ,fc-l,0 fc ) and (i = ((k - l) k+1 ,l k ). Then a vertex of GT(A, n) C X 2 k+i 
contains entries with denominator k. 



2 Proof of the Main Result and its Consequences 

Proof of Theorem \1.5l Suppose that is the tiling of a GT-pattern x in the 
GT-polytope GT(A,/i) C X n . Let s be the number of free tiles in 3? . Let 
(e' 1 ), . . . ,e( d >) be a basis for ker^4^a. Because we can scale the basis by any 
nonzero scalar, we can assume that 

|efc | < l/2min{|a; ii:)1 - x l2h \ : x lin ^ x l2j2 }, for 1 < m < d, 1 < k < s, 
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where e k is the kth coordinate of e^ m \ 

Let H C X n be the linear subspace of X n such that H + x is the affine span 
of the minimal face of GT(A, /z) containing x. Define a linear map <p : kev — » 

X„ by <^(e( m )) = y( m) , where 

( m ) J e^ if is in the free tile Pk of 
y 1 if is not in a free tile of J 2 *. 

(See Example EH|) Thus, x + y( m ) is the result of adding e£ to each entry in 
the fcth free tile of x for 1 < k < s. 

The claim is that (y' 1 -*, . . . , y^) is a basis for H. First, since the el 's 
are sufficiently small, x ± y( m > is a GT-pattern. Moreover, y{^ = 0, y 2 ( ™ } = 
for 1 < i < n, and each row-sum of y( m > is 0. This last fact is true because 
€ (m) g ker and the row-sum is, by construction, the same as the dot product 
of e^ 1 ™' with the matrix Ags. Taken together, these properties yield that x ± 
yW £ GT(\,[i). That is, x + y( m ) and x - y( m ) are the endpoints of a line 
segment contained in GT(A, /i) that contains x in its relative interior. This 
establishes that yW , . . . , y^ E H. 

That y W , . . . , y( d ) are linearly independent clearly follows from the fact 
that e* 1 ', . . . , are linearly independent. Thus, it remains only to prove that 
y^, . . . span H. Suppose that y S H, and assume that y is scaled by a 
nonzero amount so that x±y e GT(A, /i). We construct an element e of ker A^> 
such that ip(e) = y. Note that 

• yij = when (i,j) is not in a free tile of 

• each row-sum of y is 0, and 

• if (ii.ii) and («2, J2) are in the same free tile of then yi 1 j 1 = Ui 2 j 2 - 

To see that this last property holds, it suffices (see Definition to examine 

the case where yi 1 j 1 and yi 2 j 2 are adjacent entries, i.e. where 

(»a. h) e {(ii + l.ii + 1), (ii.ii + 1), (ii - 1, ii - 1), (ii.ii - 1)}. 
Since x ± y is a GT-pattern (see Definition we must have either 

x iljl ~t~ Uiljl — x ilh "t" J/i2j2 a nd ^iiii — — S'ijja — Viiil 

or 

^iljl ^ Vii'ji — Vilil a nd ^iljl ~ Viiji — x ilh ~ Villi- 

But since (ii.ii) and (12,32) are in the same tile of we have a;^^ = Xi 2 j 2 . 
Thus, in either case, we can subtract the x entries from both sides, yielding 
Villi = Vi-ih . as claimed. 



G 



For 1 < k < s and for each in the free tile put = yij. Let 

e = (ei, . . . , e s ). Then, from the conditions on y given above, e £ keiA^ and 



(p{e) — y. Hence, the coordinates of e with respect to the basis (e 
of ker A & will also be the coordinates of y with respect to (y 



(i) 



Ad) 



(1) 



particular, (y^\ 



is a basis for H, as claimed. 



) 

yW). In 
□ 



Example 2.1. Let x be the GT-pattern 
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from Figure |21 We explicitly apply to x the constructions in the proof of Theo- 
rem ^3] This GT-pattern has tiling matrix 
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Therefore, x lies in a 2-dimcnsional face of 

GT((6,5,3,2,0),(4,1,4,5,2)). 
Applying the map <p from the proof to (e*- 1 -*, e^ 2 -*) yields 































and 



y (2) = 



From the proof just given, the affine subspace affinely spanned by the face 
containing x is affinely spanned by {x, x + y*- 1 ) , x + y( 2 ) }. 
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The machinery of tilings allows us easily to find nonintegral vertices of GT- 
polytopes by looking for a tiling with a tiling matrix satisfying certain properties 
given below. Then the tiling can be "filled" in a systematic way with the entries 
of a GT-pattern that is a nonintegral vertex. 

Lemma 2.2. Suppose that & is a tiling with s free tiles such that A^> has 
trivial kernel. Then the following conditions are equivalent: 

1. is the tiling of a nonintegral vertex x of a GT-polytope in which q gN is 
the least common multiple of the denominators of the entries in x ( written 
in reduced form); and 

2. there is an integral vector £ = . . . , £ s ) such that Agn^ = O(modq) and 
such that, for some k £ {1, . . . , s}, gcd(^, q) = 1. 

Proof. [(1) =>■ (2)] Suppose that x is a nonintegral vertex in which q is the 
least common multiple of the denominators of the entries. For each entry Xij, 
1 < i < j < n, let pij — qxij. Let Pi, . . . , P s be the free tiles of 3 s , and define 
£ = by £fc = p^ for some (i,j) € Pk (all values of p^ are equal 

within a tile). Since x has entries with denominator q, we have that, for some 
k G {1, . . . , s}, gcd(£fc, q) = 1. Moreover, since each row-sum of x is an integer, 
we have that, for fixed j G {1, . . . , n}, 

q divides ^ p tj = ^ a jk ^ k . 

l<k<s l<k<s 

(iJ)eP fc 

Therefore, Ags£ = 0(mod(j). 

[(2) =^ (1)] 3? is given to be a tiling, so some GT-pattern x with rational 
entries has tiling & . If necessary, multiply x by some integer to produce an in- 
tegral GT-pattern x with tiling 3? . Choose £ = (£i, - . . , £*) satisfying condition 
(2) such that < £i, . . . , £ s < q. Define y G X n by 

{£,k/<l if (hj) is in the free cell P k of 
if (i,j) is not in a free cell of 3? . 

Then x = x + y satisfies condition (1). □ 

Now we are ready to give the details of the proof of Theorem 1 1.71 In particu- 
lar, Propositions 12 .31 and 12 .41 settle the Berenstein-Kirillov conjecture. Proposi- 
tion l2.3l has also been proven by King et al. [S] using the hive model of Knutson 
and Tao We give here a "tiling" proof. 

Proposition 2.3. When n < 4, every GT-polytope in X n is integral. 
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Proof. Note that it suffices to prove the n = 4 case since there is a natural 
embedding X n °-s- X n+ \ defined by x i— > x, where 

_ Jo ifl<t = j<n + l, 
1 Xij-i if 1 < i < j < n + 1. 

Suppose that x S X4 is a vertex. Then, by Corollary 11.61 the associated 
tiling matrix Age has trivial kernel. Therefore, Ag> is cither a2xlora2x2 
matrix. Note also that the first and last nonzero entries of each column of 
a tiling matrix associated with a GT-pattern must be 1. Therefore, Age is a 
0/1-matrix. 

If Age is 2 x 1, then the only possibilities are 



Age = 



, Age = 



Age = 



In each case, there exists no vector £ ^ O(modg) such that Ag>£ = O(modq) for 
q > 1, so Lemma [2.21 implies that the entries of x are integral. On the other 
hand, if A@> is 2 x 2, then del Age G { — 1, 1}, i-e., gcd(det Ags, q) = 1 for q > 1. 
Therefore, Age, considered as a linear operator on Z/gZ x Z/gZ, is invertible 
for q > 1, so, by Lemma \'2. 21 x is integral. □ 

Now we show that nonintegral GT-polytopes exist in X n for each n > 5. 
Moreover, by choosing n sufficiently large, we can find GT-polytopes in which 
the denominators of the vertices are arbitrarily large. 

Proposition 2.4. For positive integer k, let A = (k k ,k — 1,0 ) and /x = 

((fc — 1) +1 ,1 ). Then a vertex of GT(X, (1) C X2k+i contains entries with 
denominator k. 



Proof. Define x^ € X 2k +i by 



n (fe) 



f (fc-j+i)(fc+i) 
fc 



Q-fc-i)(fc-i) 



if 1 <i=j < fc + 1, 

if 1 < i < j < k + 1, 

if fc + 1 < j < 2k + 1 and 1 < i < j - k, 

if fc + 1 < j < 2k + 1 and j - k < i < k, 

if fc + 1 < j < 2k + 1 and j = fc + 1, 

if fc + 1 < j < 2k + 1 and fc + 1 < i < 2k 
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Figure 3: An infinite family of counterexamples to the Berenstein-Kirillov con- 
jecture 



(See Figure|31) Then x( fc ) S GT(A,/i). The tiling matrix associated with is 



A 



1 
1 



fc-1 
k 
fc-1 



















0' 










1 
1 



Since del Age = fc, x( fc ) is a vertex of GT(A,/x). 



□ 



Proposition 12.41 explicitly constructs counterexamples to the Berenstein- 
Kirillov conjecture in X n where n > 5 is odd. Counterexamples with even n > 6 
may be constructed from these using the embedding X n X n +i given in the 
proof of Theorem 12.31 Less trivial examples with even n may be constructed 
using other tilings. 

It is interesting to note that if we fix the size of the GT-patterns, then 
arbitrarily large denominators in the vertices of GT-polytopes do not exist. To 
see this, we observe that Lemma 12.21 says that if x is a nonintegral vertex in 
which q appears as a denominator, then the tiling matrix A^< has trivial kernel 
as a linear operator W — > M™ -2 (since x is a vertex), but A^> has nontrivial 
kernel when considered as an operator (Z/qZ) s — > (Z/gZ)™ -2 . Moreover, this 
nontrivial kernel contains a vector in which one of the coordinates is a unit 
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in Z/gZ. This last condition implies that each s x s submatrix of Ag? has 
determinant equal to modulo g. 

Proposition 2.5. For fixed n, the numbers that may appear as denominators 
of entries in vertices of GT-polytopes in X n are smaller than (n — 1)( 2 ) _n_1 . 

Proof. Fix n G N. Since only finitely many partitions of {{i,j) G Z 2 : 1 < i < 
j < n} exist, there is an upper bound on the set 

J l | m is the determinant of a square row submatrix 1 
1 ' of the tiling matrix of some GT-pattern x G X n J ' 

Let iV be an upper bound on this set. The claim is that no GT-polytope in 
X n has a vertex with denominators greater than N. 

Let q > N be given. Suppose that x G X n is a vertex. Let s be the number 
of free tiles in x, and let Agg be the tiling matrix of x. Then no s x s submatrix 
of Age has determinant greater than or equal to q. Moreover, by Corollary 
11.61 some s x s submatrix of A& has nonzero determinant. Therefore, this 
s x s submatrix has determinant not equal to modulo q. However, in the 
remarks preceding this proposition, we noted that if x is a vertex in which q is a 
denominator of one of the entries, then every s x s submatrix has determinant 
equal to modulo q. This proves that N is a bound as claimed. 

Our second claim is that N is no more than (n — 1)( 2 ) _n_1 . We need to 
bound the s x s subdeterminants. In a tiling matrix all entries are nonnegative 
integers, so we know that the spectral radius of eigenvalues possible for such 
a matrix is bounded above by the maximum possible sum of entries along a 
row |llj . A row sum of the tiling matrix cannot be more than the number of 
elements in a middle row of a GT-pattern, and this is n — 1 for the longest 
row. On the other hand, how big can s be? It is no bigger than the number of 
entries in the middle of the GT-pattern, which equals ("J 1 ) — n — 1. Since the 
determinant is less than the product of the norms of the eigenvalues, we get the 
desired bound. □ 

To conclude this paper we present another proof of the following result: 

Proposition 2.6. Given the GT-polytope GT(\, fi) C X n , the Ehrhart counting 
function /(to) = # (GT(mA, m/x) n z( 2 )j is a univariate polynomial. 

Proof. It is well-known, from Ehrhart 's fundamental work, that /(to) must be 
a quasipolynomial. This means that there exist an integer M and polynomials 
go,9ii ■ ■ ■ ,9m— l such that /(to) = gi(m) if to, = i (mod M) (see details in 
Chapter 4 of J7j). So it is then enough to prove that, for some large enough 
value of m, a single polynomial interpolates all values from then on, because 
then the g^s arc forced to coincide infinitely many times, which proves that they 
are the same polynomial. 
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We use the algebraic meaning of /(to) as the multiplicity of weight n/j, in the 
irreducible representation V n \ of g[ n C. The well-known Kostant's multiplicity 
formula (see page 421 of JU|) gives that 

f(m) = J2 (-l) l(a) K(<7(m\ + 8)- m\x - S), (*) 

ues„ 

where K(b) is Kostant's partition function for the root system A n , 1(a) denotes 
the number of inversions of a, and 8 is one-half of the sum of positive roots in 
An- 

Kostant's partition function is what combinatorialists call a vector partition 
function jT^j. More precisely, K(b) is equal to the number of nonnegative integral 
solutions a; of a linear system Ax = b. The columns of A are exactly the positive 
roots of the system A n . Because the matrix A is unimodular the counting 
function K(b) is a multivariate piecewise polynomial function. The regions 
where K(b) is a polynomial are convex polyhedral cones called chambers |19| . 
The chamber that contains b determines the polynomial value of K(b); in fact 
it is the vector direction of b, not its norm, that determines the polynomial 
formula to be used. 

In formula (*) the right-hand side vector for Kostant's partition function 
is b = a(m\ + 8) — (m/t + 8). As m grows, we might be moving from one 
chamber to another. Our claim is that, from some value of m on, the vectors 
a(m\ + 8) — (m/i + 8) are inside the same chamber. To see this, note that in 
the expression (*), fi, A, and 8 are constant vectors. For a given permutation 
a, the vector direction a(mX + 8) is closer and closer to that of tr(A) when m 
grows in value. Similarly, the vector direction of m/i + 8 approaches that of 
(i when m grows. Thus, the direction of o~(m\ + 8) — (to// + 8) will approach 
the direction of b' = A + fi along a straight line. For sufficiently large to, the 
vectors o~(m\ + 8) — (m/i + 8) are contained in the same chamber, then a single 
polynomial gives the value of K(b'). 

Finally, we have that, for all values of to greater than some M, the formula 
(*) represents an alternating sum of polynomials in the variable to. Therefore 
/(to) is a polynomial for all m greater than M, exactly as we wished to prove. 

□ 
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